Search for: All records

Medical systematic reviews play a vital role in healthcare decision making and policy. However, their production is time-consuming, limiting the availability of high-quality and up-to-date evidence summaries. Recent advancements in LLMs offer the potential to automatically generate literature reviews on demand, addressing this issue. However, LLMs sometimes generate inaccurate (and potentially misleading) texts by hallucination or omission. In healthcare, this can make LLMs unusable at best and dangerous at worst. We conducted 16 interviews with international systematic review experts to characterize the perceived utility and risks of LLMs in the specific context of medical evidence reviews. Experts indicated that LLMs can assist in the writing process by drafting summaries, generating templates, distilling information, and crosschecking information. They also raised concerns regarding confidently composed but inaccurate LLM outputs and other potential downstream harms, including decreased accountability and proliferation of low-quality reviews. Informed by this qualitative analysis, we identify criteria for rigorous evaluation of biomedical LLMs aligned with domain expert views. 

We developed a virtual agent that motivates church-going users to change their health behavior by telling existing cultural narratives that have high relevance with the counseling topic in an engaging way. We evaluated this agent in a between-subjects experiment where participants interacted with an agent that counseled them on nutrition either without a story, with a story but told in a neutral speech style, or with a story using dramatic delivery inspired by church sermons. We found that interaction with either one of the storytelling agents leads to a significantly greater change in confidence to engage in the target behavior of healthy eating than interacting with a non-storytelling agent, demonstrating the efficacy of stories in health counseling by virtual agents. 

We developed a virtual agent that motivates church-going users to change their health behavior by telling existing cultural narratives that have high relevance with the counseling topic in an engaging way. We evaluated this agent in a between-subjects experiment where participants interacted with an agent that counseled them on nutrition either without a story, with a story but told in a neutral speech style, or with a story using dramatic delivery inspired by church sermons. We found that interaction with either one of the storytelling agents leads to a significantly greater change in confidence to engage in the target behavior of healthy eating than interacting with a non-storytelling agent, demonstrating the efficacy of stories in health counseling by virtual agents. 

Keeping users engaged with mHealth applications is important but difficult to achieve. We describe the development of a smartphone-based application designed to promote health and wellness in church communities, along with mechanisms explicitly designed to maintain engagement. We evaluated religiously tailored techno-spiritual engagement mechanisms, including a prayer posting wall, pastor announcements, an embodied conversational agent for dialogue-based scriptural reflections and health coaching, and tailored push notifications. We conducted a four-week pilot study with 25 participants from two churches, measuring high levels of participant acceptance and satisfaction with all features of the application. Engagement with the app was higher for users considered to be more religious and correlated with the number of notifications received. Our findings demonstrate that our tailored mechanisms can increase engagement with an mHealth app